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Abstract 

This study reviews theories supporting the construction of student knowledge 
and found they rely upon theoretical operations with few pragmatic 
applications. A new paradigm of learning needs to emerge which focuses on 
how students actually construct their own knowledge, based on 
reconstructing current/past knowledge or experience, and their interpretation 
of the world. Such a paradigm implies ways that would enable educators to 
facilitate students ’ problem solving, and gears to help students construct 
hypotheses, select and transform information, and finally make decisions. 
Implications for designing effective narrative cases for ill-structured problem 
solving are discussed. 
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First heading 
Introduction 

Bereiter and Scardamalia (1993) identified problem solving as an important aspect of professional 
practice. Professionals apply their problem solving skills to the ill-structured nature of their work 
(Schon, 1987), and ill-structured problems are prototypical of problems that are often found in 
real-world practices. Much research has identified and developed methods for supporting and 
facilitating organizational ill-structured problem solving styles in education (e.g., Ge & Land, 
2004). Mayo (2002) formulated a hypothetical case narrative which asked students to identify the 
theory that most closely fits with a provided case situation and provide a brief, explanatory 
rationale for their selection. Mayo’s (2002) study found students who received case narrative 
problems improved their conceptual understandings more significantly than those who did not 
receive any. As the students engage in real-life situations, they are able to evaluate critically and 
generate as many alternatives as possible rather than process information primarily on emotional 
and intuitive levels (Kreber, 2001). However, these researchers have not yet addressed the issue of 
designing effective cases in terms of ill-structured problems to facilitate students’ problem solving 
skills; rather than describe the process of solving the problems in their publications, researchers 
have focused on outcomes. 

Literature Review 

In case-based instruction, real or fictional problem situations are implemented for students’ 
learning. Previously, case-based instruction has been found to encourage team work, facilitate the 
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development of reflective thinking and deeper conceptual understanding (Heath, 1999). Likewise, 
case-based instruction has been shown to facilitate meaningful learning by bridging the gap 
between theory and practice (Ertmer & Russell, 1995). 

Osana, Tucker, and Bennett (2003) investigated the nature of adolescent reasoning on the issues of 
reform-oriented social studies through solving cases. In their study, eighteen participants from 10 th 
grade were asked to allocate money to the candidates and to write their justifications for the 
allocations. Think-alouds and interviews were analyzed, and the results found that adolescents are 
more likely to engage in constructive or sense-making activity while reasoning through ill- 
structured problems. 

Much research has focused on students’ learning outcomes on problem-based instruction, little has 
yet been conducted on the effectiveness of the problem-based tutorial as an instructional strategy. 
Gallagher, Stepien, and Rosenthal (1992) studied the strategies that helped students problem solve. 
They found that problem finding during the problem solving steps was significantly increased and 
implied an increased recognition of problem finding is especially important since this is the 
component of problem solving considered critical for creative productivity. This increase in the 
presence of this problem solving step is a promising trend and justifies additional refinement as an 
instructional alternative for talented students. However, the participants in Gallagher et al’s (1992) 
study were talented students, so their findings may not apply to other students if we use the same 
problem-based instruction as a technique. Therefore, problem based instruction is in need of 
additional research. 

Researchers have investigated the approaches students use to solve problems. There are two 
common ones that have been offered to increase productions or generations of solutions to ill- 
structured problems: incubation (Wallas, 1926) and brainstorming (Osborn, 1953). In addition to 
incubation and brainstorming, the use of hierarchy is also a common approach with other 
researchers. Butler and Thomas (1999) conducted a study involving ninety-eight participants who 
were randomly assigned to one of the three experimental conditions (hierarchical, brainstorming, 
and incubation). The problem was given to a group of four or five people and they were given a 
brief description that varied according to the condition they were in. Their study found that the 
active methods of brainstorming and hierarchical generation produced more ideas than the passive 
technique of incubation (Butler & Thomas, 1999). However, the effectiveness of these strategies is 
inconsistent, and they can cause students to overlook some related solutions and associates. 

In addition to research on various strategies for problem solving, researchers have been interested 
in the assessment of complex problem solving contexts. To find out whether skills required for 
solving well-structured and ill-structured problems generalize to each other. Wood (1983) 
discussed the issue of problem space. Problem space is a unit of human goal-oriented symbolic 
activity. Wood (1983) stated that with a problem space framework, different types of problem 
solving situations can be understood by outlining symbolic structures and then by describing the 
paths, goals, and rules for achieving the goal state. Furthermore, Wood (1983) claimed that the 
processes used to solve such problems are not the same as those used to solve well-structured 
problems. This has greatly influenced how the instructional designers should design cases for ill- 
structured problem solving. 

Jonassen and Kwon (2001) hypothesised the functional content and patterns of communication 
would vary between well and ill-structured problems and by mode of communication. They 
presumed functional content and patterns during computer conferences would be more purposive 
and task-oriented than during face-to-face discussion, and ill-structured problems should require 
more complex and less predictable patterns of communication. The instruments of surveys of on 
Surveys of students’ perceptions, and students’ conversations were used in their study. In terms of 
problem solving functions, every communication act was classified as a problem definition, 
orientation, solution development, non-task, simple agreement, or simple disagreement. Students’ 
problem solving patterns were classified and transformed into one of eight problem solving phases 
including problem analysis, problem critique, orientation, criteria development, solution 
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development, solution approval, solution critique, and non-task. They found learners in the 
computer-mediated communication group participated interactively in the group learning process, 
which reflected multiple perspectives by the greater number of agreements and disagreements. 

From previous research, we know that the notions of well-structured and ill-structured problems 
are different (e.g., Simon, 1978; Wood, 1983); thus, the examination of the role of communication 
patterns while solving ill-structured problems becomes important. 

Research Questions 

This study examined the following questions which may help us to understand how to design 
effective ill-structured problems. 

(1) What are the patterns for solving ill-structured problems? 

(2) What are the implications of designing authentic case narratives that reflect real-world ill- 
structured problems? 

Methodology 

Participants and Procedure 

The study took place in a medium-sized, southwestern university. Two voluntary graduate -level 
educational psychology students participated in this study. The researcher distributed a 
hypothetical case scenario about an instructional design related problem adapted from Ertmer and 
Quinn (1999). The participants were first instructed to perform think-aloud protocols using the 
think-aloud process developed by Ericsson and Simon (1993). During the problem solving, 
participants performed think-aloud protocols on solving the problem. The use of think-aloud 
methodology was efficient in this study because it provided a window for uncovering the 
psychological mechanisms and knowledge structures underlying human problem solving activities 
(Yang, 2003). 

After the problem solving task, a semi-structured interview was conducted for an in-depth 
investigation of students’ thinking processes. A one-on-one interview is practical for investigating 
the research questions because students are more likely to express their thoughts to open-ended 
questions asked in a private setting than in focus groups or other more exposed settings (Creswell, 
1998). As a result, the more we can examine students’ thinking processes, the more we know 
about how they solve ill-structured problems. 

Design 

Case study data analysis techniques (Miles & Huberman, 1994) were used to process data, 
generate emerging themes, and refine the existing framework. This study also intended to validate 
existing theories on ill-structured problem solving as the result of qualitative analysis. 

Data Analysis 

Data analysis was carried out at three primary levels: (1) initial data from think-aloud protocols 
and interviews were organized according to the research questions; (2) segments of the transcript 
data were coded and categorized in the transcript margins to index and cluster data; (3) schemes 
were generated and grouped for the research questions. The coding schemes developed were 
presented in Table 1. Peer review and member checking were also conducted to verify the 
reliability and validity of the qualitative findings. 

Preliminary Findings 

A summary of findings was reported below to answer the two research questions concerning how 
people solve ill-structured problems as well as the implications for designing them. The 
qualitative findings revealed that subjects presented general levels of problem solving skills. 
Subjects showed more or less organized responses in different problem solving stages, which 
affected how the subjects used different approaches on solving the problem. In the problem 
representation, participants selected or adjoined critical information needed and excluded 
superfluous distractions from the complicated problem situation so as to gain a clear representation 
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of the ill-structured problems. For example, Jenny identified the key issues (i.e., the web can 
provide students with various resources and historical background) that were critical to help her 
understand the case and further the analysis plan. In seeking information, participants made 
inferences from their prior experiences to explore a possible path for arriving at the solutions. 

They tried out different strategies to gauge information. In finding alternative solutions, 
participants chose solutions dependent on the differences among their domain knowledge and 
problem solving expertise. In the analysis, participants used strategies such as self-monitoring and 
self-evaluation to guide their plans of action. In the solution argument, participants performed 
metacognitive thinking to guide their reasoning and decision-making. 

From the interview data, one of the subjects responded that the most difficult process while 
working on the problem was that there was not enough information for her to solve the case. She 
was tied up with limited existing information provided in the case and was aware of the gap 
between the presented information and problem solution. The second subject responded that she 
focused on trying to find out what her role was in the case so that she could provide better and 
more resources to the existing information. 

Our qualitative data findings revealed important implications for the design of ill-structured 
problems. In the problem representation, more external instructional supports should be provided 
to help students clarify the problem statement. The ill-structured problems should include 
concepts, rules, and principles in specific domains, and represent associated premises from similar 
situations. In the seeking information and solution alternative, the ill-structured problems should 
devise different paths for primary solutions and alternative solutions. In the analysis and solution 
argument, the ill-structured problems should embed the opportunities for students to look back and 
justify their solutions to the problems they have identified. 

Conclusion 

Researchers have discussed the importance of ill-structured problems on promoting students’ 
conceptual understanding and knowledge transfer (e.g., Ge & Land, 2004). This study investigated 
how people solve the ill-structured problem on the domain of instructional design. When solving 
the problems, students relied mostly on their prior experience and knowledge to guide their 
thinking and approaches in arriving at the solutions. This study provided in-depth investigation of 
the processes, such as problem representation, seeking alternative solutions and solution argument 
that learners used to solve problems. Domain knowledge and the use of cognitive strategies (i.e., 
self-evaluation and monitoring) are two critical components of successful problem solving. 

Teachers who are interested in incorporating ill-structured problems into the curriculum should be 
cautious of the difficulties that our students will run into when no instructional supports are 
provided. This study offers criteria that teachers can use to guide the design and understanding of 
ill-structured problems. Although the design of the ill-structured problem for this study is on 
instructional design domain, future research examines other domains of problem should shed 
lights on the effectiveness of ill-structured problem solving. 
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Table 1 : Coding scheme, examples, and analysis from the data 


Themes/Codes 

Participants’ 

Examples 

Analysis 

Problem 

representation 

Jenny 

“She wants to have a website because there are 
some advantages. First, the web can provide 
students with various resources and historical 
background. Students can get more information 
than in the textbook. More and also current. 1 
agree with that. Second, the web can provide 
students with easy access to various visuals, such 
as pictures, audio, and video clips, which students 
can study any time of the day. Yes, 1 agree with 
that.” 

Jenny reported and summarized the issues 
as described in the case. Her report toward 
the case is often verbatim. It appears that she 
paid little efforts to consider the extent to the 
heart of the case. 


Cathy 

“1 guess first thing we need to do is to find out 
what the exact objectives are. That would be kind 
of identify the problem, determine what the 
outcomes, goals, main goals and the specific 
objectives. That’s always the good starting point.” 

Cathy identified and categorized the central 
issues in a case. 

Seeking 

information 

Jenny 

“1 need to know how much money we can use 
because if it is enough, she can hire someone for 
designing and developing the website in addition 
to me.” 

Jenny tended to focus on what she did not 
know and made the inferences from unknown 
information. 


Cathy 

“Huh, need to know specifically what literature that 
she is going to teach. It says Shakespeare, but 
does that the only thing is going to be covered, or 
there are other topics. There always be a 
creditability issue” 

Cathy took a more focused on what she 
already knew from the case. 

Alternative 

solution 

Jenny 

“First of all, 1 need to know what Dr. Lou needs 
regard the content of the website. What students 
want to have or see for the website? What else? 
As 1 thought previously, it is very important, what 
is the role of the website for her instruction. 
Whether she wants to use it in her class like she 
wants to have students assign to work during the 
class or using the website as the textbook in 
addition to her regular instruction in the class. And 
after 1 get those information about the content, the 
type of website, the available facility, special 
software or hardware that we need. Then 1 can 
draw simple kind of design for website.” 

Jenny identified and proposed solutions to the 
case. She also considered for how the issues 
might be related to each other. 


Cathy 

“She got the small grant, find how much the grant 
is for, and determine how many hours we estimate 
this is going to take. Make sure we have basic of 
hours to determine hours and costs may take. To 
make we have enough and then find out if we 
need more money from some other places or 
possibly cutting certain elements because you do 
not have enough funding to finish it.” 

Cathy identified several suggestions to the 
solutions and linked the suggestions in a 
more coherent way. 

Analysis 

Jenny 

“Dr. Lou has to tell me the website. Do 1 just 
design one time or 1 need to update. It’s very 
important otherwise 1 cannot estimate the labor of 

Jenny addressed the different issues to help 
herself modify the solutions. 


1 Pseudonyms are used to protect the identities of the participants. 
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time which is required for me to make a plan. If I 
want to create such a website, I also want to make 
it like very attractive. I also need to know about 
what students want. How to attract students? 

What kinds of stuff they want to know related to 
classical literature such pictures, hometown, 
stories, articles, or how to access some specific 
sites? Exactly what they want or what students 
want to see on the website. First, I have to talk to 
Dr. Lou. Then probably I will request her to have 
me interview students in her classroom. Then I 
can determine what they want. If it is not feasible, 
at least I will request Dr. Lou to provide the survey 
so students can quickly respond.” 

Cathy “Because I think it’s could make the experience Cathy proposed the solution in an advisory 

richer as supposed to just sitting and reading and tone, 
huh, the thing that might be good where they get 
more out of it would be have more interactive, say, 
discussion board, something on blackboard or 
something where they could discuss the themes 
and everything among other students.” 

“Once we make final plan, if it is possible before 
just starting to create the website. I am going to 
develop a kind of demonstration, like simple one 
to provide people brief idea of what the website 
looks like. I want to show it to Dr. Lou and some of 
her students. Like trial version. To see what they 
like, what they do not like, and what they need 
which is not in the website. 

Then after I get information, I can make a second 
plan for revision. And also during that time, I also 
revise the plan for cost. And go ahead make a 
final one. Once I complete final one, Dr. Lou can 
use it as the trail. If it is completed, there should 
be some problems that I can revise. Hum, I still 
need the feedback for that.” 


Solution Jenny 

argument 


Jenny went beyond the focus on the case, 
and she also considered other issues that 
may be important. 


Cathy “And of course running the evaluation, that is one Cathy recalled her prior experience from the 

of the important thing that I had when developing similar context. 

(when I did my master internship that I developed 
a series of labs. That’s one of final and important 
step was the formative evaluation. We had some 
student go through it, try out, and give us 
feedback. Based on the feedback, we go back 
and change a few things to make it even better. 

That’s pretty much it.” 
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